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Abstract 

We determine the rate region of the quadratic Gaussian two-encoder 
source-coding problem. This rate region is achieved by a simple archi- 
tecture that separates the analog and digital aspects of the compression. 
Furthermore, this architecture requires higher rates to send a Gaussian 
source than it does to send any other source with the same covariance. 
Our techniques can also be used to determine the sum rate of some gen- 
eralizations of this classical problem. Our approach involves coupling the 
problem to a quadratic Gaussian "CEO problem." 

Keywords: multiterminal source coding, vector quantization, Gaussian 
sources, rate region, worst-case source, remote source, CEO problem. 

1 Introduction 

This paper addresses the quadratic Gaussian two-encoder source-coding prob- 
lem, the setup for which is depicted in Fig. 1. Two encoders observe different 
components of a memoryless, Gaussian, vector-valued source. The encoders, 
without cooperating, compress their observations and send messages to a single 
decoder over rate-constrained, noiseless channels. The decoder attempts to re- 
produce both observations, subject to separate constraints on the time-avcraged 
expected squared error of the two estimates. We seek to determine the set of 
rate pairs (i?i, R2) that allow us to meet a given pair of target distortions. Wc 
call this set the rate region. Of course, this problem can also be formulated for 
general sources and distortion measures. Our focus on the quadratic Gaussian 
case is motivated by its fundamental nature, its importance in applications, and 
its well-known extremal properties. 
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This problem is naturally viewed as a quadratic Gaussian version of Slepian 
and Wolf's problem [1]. Slepian and Wolf studied the problem in which the 
source is discrete and the decoder must reproduce it with negligible probability 
of error. Their celebrated result characterizes the rate region for this setup. One 
consequence of this characterization is that permitting the encoders to cooperate 
or view each other's observations does not offer any advantage, at least as far 
as the sum rate is concerned. 

There is a natural way to harness Slepian and Wolf's result in the Gaussian 
setting. Each encoder first vector quantizes (VQs) its observation using a Gaus- 
sian test channel as in single-encoder rate-distortion theory. This results in two 
correlated digital messages, which are suitable for compression via Slepian- Wolf 
encoding. The decoder decodes the quantized values and estimates the source 
by computing a conditional expectation. This approach separates the analog 
and digital aspects of the compression, as shown in Fig. 2. 

Our main result is an explicit characterization of the rate region for this 
problem. This result has three notable consequences: 

(i) The architecture depicted in Fig. 2 is optimal. 

(ii) This architecture requires higher rates to send a Gaussian source than it 
does to send any other source with the same covariance. In particular, a 
Gaussian source has the smallest rate region for a given covariance. 

(Hi) Unlike in the Slepian- Wolf problem, here decentralized encoding requires 
a strictly higher rate compared to centralized encoding. 

The problem of determining the rate region for this setup has been open 
for some time [2]. Early work [2, 3] used the architecture described above to 
prove an inner bound. Zamir and Berger [4] showed that this inner bound 
is asymptotically tight in the low-distortion regime, even if the source is not 
Gaussian. Oohama [5] determined the rate region for the problem in which only 
one of the two distortion constraints is present. By interpreting this problem 
as a relaxation of the original problem, he obtained an outer bound for the 
latter. He showed that this outer bound, when combined with the inner bound, 
determines a portion of the boundary of the rate region. As a result of his work, 
showing that the inner bound is tight in the sum rate suffices to complete the 
characterization of the rate region. This is shown in the present paper. 

Our approach is to lower bound the sum rate of a given code in two different 
ways. The first way amounts to considering the rate required by a hypothetical 
centralized encoder that achieves the same error covariance matrix as the code. 
The second way is to establish a connection between this problem and the 
quadratic Gaussian "CEO problem," for which the rate region is already known. 
For some codes, the cooperative bound may be tighter. For others, the CEO 
bound may be tighter. Taking the maximum of the two lower bounds yields a 
composite lower bound that is sufficiently strong to prove the desired result. 

The next section contains a precise formulation of the problem and a state- 
ment of our main result, Theorem 1. In Section 3, we describe the separation- 
based compression architecture. There we also discuss the worst-case property 
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of the Gaussian distribution. We provide the necessary background on the CEO 
problem and some other preliminaries to the converse proof in Section 4. The 
converse proof itself is contained in Section 5. In Section 6, we show how the 
converse proof technique can be used to determine the rate region for a more 
general version of the problem in which the decoder aims to reproduce certain 
linear combinations of the source components. In Section 7, we show how the 
proof technique can be extended to handle the case of more than two sources, if a 
certain symmetry condition holds. Section 8 contains some concluding remarks. 

We use the following notation. Boldface, lower case letters (fi) denote vec- 
tors, while boldface, upper case letters (D) denote matrices. Lightface letters 

R) denote scalars. Whether a variable is deterministic or random should be 
clear from the context. 



2 Problem Formulation and Main Result 

Let {(y™ (i), y% («))}"= i De a sequence of independent and identically distributed 
(i.i.d.) Gaussian zero-mean random vectors. Let 



Ky = 



1 P 
P 1 



(1) 



denote the covariance matrix of (y"(l), y% (1))- We use to denote 

{y?(i)}2=i, 

yf(J : k) to denote 

y n (i) to denote 2/^(*)), y™ to denote {(y™ (i), y% etc. Analogous 

notation will be used for other vectors that appear later. 

The first encoder observes y™, then sends a message to the decoder using a 
mapping 

fi n) :R n ^{l,...,M[ n) }. 

The second encoder operates analogously. The decoder uses the received mes- 
sages to estimate both and using mappings 

<p™ : {l, . . . , M^} x {l, . . . , M 2 (,l) } ^ R» j = 1, 2. 

Definition 1 (Quadratic Gaussian Two-Encoder Source Coding Problem). A 
rate- distortion vector i?2, di, cfe) is strict-sense achievable if there exists a 
block length n, encoders f[ and > an d a decoder (ip^ , ^2 ) sucn that 1 

Rj > -logA/] ,l) for allj in {1,2}, and 

1 " .1 ( 2 ) 



1 A\\ logarithms in this paper are base two. 



3 



^ = <pf(f[ n \yi),fi n \y^)) je{i,2} 



wher 



Let 7ZD* denote the set of strict-sense achievable rate- distortion vectors. We 
define the set of achievable rate- distortion vectors to be the closure, TZT>* , of 

TZV*. Let 

TV{d u d 2 ) = {(Ri,R 2 ) ■ {Ri,R2,d 1 ,d 2 )eTlV*Y 

We call 1Z*(-, •) the rate region for the problem. The (minimum) sum rate for 
a given distortion pair {d\,d 2 ) is defined to be 

iid{Rx + R 2 : (Rx,R 2 ) G U*(di,d 2 )}. 

We note that there is no loss of generality in assuming that E[y\] = E\y£\ = 
1, since the observations and the estimates can be scaled to reduce the general 
case to this one. By similar reasoning, we may assume that p > 0, i.e., that the 
observations of the two encoders are nonncgatively correlated. Since the two 
extreme cases p = and p = 1 can be handled using existing techniques, we 
will assume throughout the remainder of the paper that < p < 1. 

We now define three sets that will be used to describe the rate region. Let 



n(di 



(R 1 ,R 2 ):R 1 > -log+ 



where log + x = max(loga;, 0). Likewise, let 



TV 2 {d 2 ) = 
Finally, let 

^sum( rf l,d2) 

where 



{R X ,R 2 ) :R 2 > ^log+ 



1 , 9 



P 2 2 



(Ri,R 2 ) :R!+R 2 > -log H 



I3(d u d 2 ) = 1 + 



(l-p 2 )^!,^) 



2did 2 




Later we will see that we can often interpret the logarithm in the definition 
of 7\L* um (-, •) as a mutual information 



\D 



where D* is the covariance matrix of the errors (j/i — yi,y 2 — y 2 ) in a sum-rate 
optimal code. Throughout the paper we assume that all distortion constraints 
(di and d 2 in this case) are positive. 



Theorem 1. For the Gaussian two-encoder source-coding problem, 

n^d^d^^niid^mzKd^n n* sum (d u d 2 ). 



(3) 
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An example of the rate region is shown in Fig. 3. The direct part of this 
result was previously known and is discussed in the next section. It was also 
previously known that the rate region was contained in the set 1Z\ {d\) r\lZ 2 (d 2 )■ 
Our contribution is a proof that the rate region in contained in lZ* UTa (di, d 2 ). 
This is provided in Sections 4 and 5. Sections 6 and 7 present some extensions 
of this result to problems with more general distortion constraints and more 
than two sources, respectively. 

3 Direct Part and Worst-Case Property 

Translating the architecture in Fig. 2 into an inner bound on the rate region is a 
straightforward exercise in network information theory. Since proofs of similar 
bounds are available [2, 6, 7, 8, 9], we provide only a high-level view of the proof 
here. 

Let lA(d\, d 2 ) denote the set of real- valued random variables U\ and u 2 such 
that 

(i) m <-> yi «-> 2/2 u 2i meaning that m, yi, y%, and u 2 form a Markov chain 
in this order 2 , and 



Then fix u in U(di, d 2 ) and a large integer n. By the proof of the point-to-point 
rate-distortion theorem, the first vector quantizer can send I(yi;ui) bits per 
sample to the first Slepian-Wolf encoder that conveys a string that is jointly 
typical with yf with high probability. Likewise, the second vector quantizer can 
use I(y 2 ;u 2 ) bits per sample to send to its Slepian-Wolf encoder a string u 2 
that is jointly typical with y 2 with high probability. 

The Slepian-Wolf encoders could view the quantized strings u" and u 2 as 
individual symbols from a digital source to be compressed [9] . They would then 
accumulate many such symbols to compress. Alternatively, one can apply the 
arguments behind the Slcpian and Wolf theorem directly to w" and u 2 [2, 6, 7, 8]. 
Either way, the decoder can recover u™ and u 2 so long as 



The decoder can then in principle compute the minimum mean-squared error 
(MMSE) estimate of y™ given u n , and (ii) above guarantees that this estimate 
will comply with the distortion constraints. By a time-sharing argument, one 
can show that the rate region is convex. This outlines the proof of the following 
inner bound. 

2 This condition is sometimes called the "long Markov chain" [4]. 



(it) E[( yj - E[ Vj \u]) 



2 }<d 3 for j e {1,2}. 



Ri > I(yi;ui\u2) 
R2 > Hy2;u 2 \u 1 ) 
Ri + R 2 >I(y;u). 



(4) 
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Proposition 1 (Berger-Tung Inner Bound [2, 3]). The separation-based archi- 
tecture achieves the rates 

TZ z (di,d 2 ) = {(Ri,R 2 ) : there exists u G U(di,d 2 ) such that 

R\ > I{yi)ui\u-z) ^ 
R2 > I{y2\u 2 \u 1 ) 
R1 + R2 >/(y;u)}. 

In particular, the rate region contains the convex hull of this set. 

It is unclear a priori how to compute this inner bound. A natural approach 
is to place additional constraints on u to create a potentially smaller inner bound 
that is amenable to explicit calculation. Let Ua(di,d 2 ) denote the set of u in 
U(di,d 2 ) such that Uj has zero mean and unit variance for each j, and there 
exists a random vector z such that 

(i) For some constants c\ and c 2 in [0, 1), 

ui = ciyi + zi 
u 2 = c 2 y 2 + z 2 , 

(ii) z is Gaussian and its components are independent, 
(Hi) z is independent of y, 
iiv) E[{ Vj - E[vM?\ < dj for all j in {1, 2}. 

We will refer to a random vector u satisfying conditions (i)-(iii) as a distributed 
Gaussian test channel or, when there is no ambiguity, as simply a test channel. 
Note that the set of distributed Gaussian test channels is parametrized by c\ 
and c 2 . 
Let 

TZc(di,d 2 ) = {(Ri,R 2 ) : there exists u g Uc(di,d 2 ) such that 

Ri > I(2/i;ui|u 2 ) ^ 

R2 > I(y2;u 2 \ui) 
R1 + R2 >/(y;u)}. 

Lemma 1. The separation-based architecture achieves TZa(di,d 2 ), which satis- 
fies 

K G {d ll d 2 ) = Kl(d 1 )r\K* 2 (d 2 )r\K* uni (d 1 ,d 2 ). (7) 

Proposition 1 immediately implies that the separation-based architecture 
achieves 1Za{d\,d 2 ), so we only need to prove the equality in (7). Later, we 
state and prove a more general version of this equality (Lemma 7 in Section 6) . 
Since one can verify directly that %\(d\), lZ 2 (d 2 ), and lZ* unl (di, d 2 ) are convex, 
it follows that TZa{di,d 2 ) is also convex. Thus, time sharing does not enlarge 
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this region. In the remainder of the paper, whenever we consider the separation- 
based architecture, we will assume that u is a distributed Gaussian test channel. 

Theorem 1 and Lemma 1 together show that TZG(di,d 2 ) equals the rate 
region. In particular, this implies that the separation-based scheme depicted in 
Fig. 2 is an optimal architecture for this problem. We note that the quadratic- 
Gaussian two-encoder source-coding problem is not unique in this respect. Prior 
work has shown this architecture to be optimal for other important problems 
as well [4, 6, 7, 8, 10, 11, 12]. 

In fact, the separation-based architecture achieves the rates 

K* 1 (di)nimd 2 )nKt um (di,d 2 ) 

even if the source is not Gaussian. Let {y™(i)}" =1 be a sequence of zero-mean 
i.i.d. random vectors, not necessarily Gaussian, with covariance matrix K y . 
We consider the same source-coding problem as before, but with the alternate 
source y in place of y. Let 1Z l (di,d 2 ) denote the inner bound obtained from 
Proposition 1. 

Proposition 2. The separation-based architecture achieves the rates 

n\(d l )fMV 2 {d 2 )r\'R k snm {d 1 ,d 2 ) (8) 

for the source y. That is, TZ l (di,d 2 ) contains this set. 

Proof. See Appendix A. □ 

Theorem 1 and Proposition 2 together imply that the separation-based ar- 
chitecture requires higher rates to send a Gaussian source than it does to send 
any other source with the same covariance. In particular, a Gaussian source 
has the smallest rate region for a given covariance matrix. This result is a two- 
encoder extension of the well-known fact that a Gaussian source has the largest 
rate-distortion function for a given variance [13, Ex. 9.7] (see Lapidoth [14] for 
a stronger version). 



4 Converse Preliminaries 

Oohama [5] determined the rate region when only one of the two distortion 
constraints is present 

n*{d 1 ,i) = n* 1 (d 1 ) (9) 

K*(l,d 2 ) = K* 2 {d 2 ). (10) 
As a consequence of his result, it follows that 

n*(d u d 2 )cnudi)mmd 2 ). (11) 

This outer bound is tight in a certain special case. Let T>c denote the set of 
matrices D such that 

D- 1 = K~ 1 +A (12) 
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for some diagonal and positive semidefinite matrix A. There is a one-to-one 
correspondence between Vq and the set of distributed Gaussian test channels. 
Specifically, D is the covariance matrix of y — £7[y|u], where u is a distributed 
Gaussian test channel with 



'5 = TTA- ^ 2 > 

and Ai and A 2 are defined by 

A 



Ai 
A 2 



As such, we will sometimes refer to D, or equivalcntly A, as a (distributed 
Gaussian) test channel. Note that the mutual information between y and u can 
be expressed in terms of D 

7(y;u) = fc(y)-My|u) 

= ilog((2^) 2 |^|)-ilog((27re) 2 |r>|) 

Let diag(2?G) denote the set of distortion pairs (d\, (I2) such that there exists 
a D in T>q with top- left entry d\ and bottom-right entry di . It is straightforward 
to verify that (d\, (I2) is in diag(2?c) if and only if 

max(di, d 2 ) < min(l, p 2 ■ min((ii, d 2 ) + 1 — p 2 )- (14) 

The set diag(2?c) is significant because if (di,d2) is not in d\&g(T>c) : then the 
rate region can be determined using existing results. 

Lemma 2. If (di,d,2) is not in diag^c), then 

Ki(d 1 )nK* 2 (d 2 )CKt um (d 1 ,d 2 ). (15) 

In particular, the rate region equals 

n^d 1 ,ch) = ni(d 1 )nn^d 2 )nn* nm (d 1 ,d 2 ). (16) 

The proof is given in Appendix B. In light of this lemma, Lemma 1, and 
(11), it suffices to show that when (di,d 2 ) is in diag(2?G), 

n*{d u d 2 ) c^* um (d!,d 2 ). 

We show this in the next section. 

Our proof uses a characterization of the sum rate of the quadratic Gaussian 
CEO problem. In the two-encoder version of this problem, encoders 1 and 2 



observe y\ and y 2 , respectively, and then communicate with a single decoder as 
in the original problem. But now y\ and y 2 are of the form 



yi = a\x + ni 
y 2 = a 2 x + n 2 , 

where x, ni, and n 2 are independent and Gaussian, and the decoder estimates 
x instead of y\ and y 2 . The distortion measure is again the average squared 
error. This problem's rate region was determined independently by Oohama [8] 
and Prabhakaran, Tse, and Ramchandran [10] 3 . Their result shows that the 
separation-based architecture is optimal for this problem. 

For our purpose, we will find it more convenient to consider the problem 
in which the decoder attempts to estimate fJ^y for some given vector fi. We 
call this problem the (j,-sum problem. For some values of fi, the /i-sum problem 
can be coupled to a CEO problem. For these values of fi, it follows that the 
separation-based architecture is optimal. 

Lemma 3. The sum rate for the /i-sum problem with pi ■ p 2 > and allowable 
distortion d equals 

inf I i log ip^j! : D G V G and fj, T Dfj, < dj . (17) 

In Appendix C, we prove an extended version of this lemma that includes a 
description of the entire rate region. Here we note some properties of T>q and 
the optimization problem (17). Recall that D is in T>q if there exists a diagonal 
and positive semidefinite matrix A such that 

D- 1 =K~ 1 +A. (18) 

This formula provides a convenient way of evaluating the off-diagonal entry of 
D in terms of its diagonal entries and p. Let us write 



D 



di d\Jd\d 2 
Q\Jd\d 2 d 2 



where 8 G ( — 1, 1). Equating the off-diagonal entries in (18) gives 

9 p 



(1 - 2 )VdId2~ 1-P 



2 ' 



(19) 



Since 2 < 1, it follows that 9 must be positive. But this quadratic equation in 
9 has only one positive root 



V(i-pT + V^2-(i-^ 

2p^did 2 



3 In fact, both works solved the problem for an arbitrary number of encoders, but this 
generality is not needed at this point. 



Thus there is no other matrix in T>q with top-left entry d\ and bottom-right 
entry d 2 . Using (20), the determinant of D can be expressed in terms of d\ and 

where /?(•,•) was defined in Section 2. The effect of the product d^d 2 on 9 is 
shown in Fig. 4. As did 2 tends to 1, 8 converges to p and D converges to K y . 
On the other hand, as did 2 tends to zero, 6 also converges to zero, i.e., the 
errors become asymptotically uncorrelated. 

Next we show that every matrix in T>q solves a /j-sum problem for some fi 
with /ii • /j<2 > 0. This fact will be used in the proof of our main result. 

Lemma 4. Let 

di 



D* 



f Vd^d 2 



be in T>q, and let 



Then D* is sum-rate optimal for the fjb 



0*VdId~2~ 
d 2 

fd[ 

-sum problem, 



i.e. 



■log 



\Ky 

ID* 



inf - log 



\J£y_ 
\D\ 



D G T>g and n Dfi* < n* 1 



(22) 



(23) 



The proof is deferred to Appendix D. It is helpful to note that if the diagonal 
entries of D* are equal, then the coordinates of /x* will also be equal. This 
fact makes the proofs that follow somewhat simpler when the two distortion 
constraints, d\ and d 2 , are equal. As such, the reader is encouraged to keep this 
case in mind as we turn to the proof of the main result. 



5 Proof of the Main Result 



Recall that we may restrict attention to the case in which (d\, d 2 ) is in diag(2?c). 
Let us now fix one such distortion pair; we will suppress dependence on [d\, d 2 ) 
in what follows. Let D* denote the element of T>q whose top-left and bottom- 
right entries are d\ and d 2 , respectively. 

Definition 2. For 9 g (-1,1), let 

di 



Do 



eVdTd~ 2 



and define 



Rcoop(0) = ^log 



\K 



eVd^ 

d 2 

1 



I-P 2 



2""° \D__ 

Let fi* be the vector defined in (22). Then let 



vi = - log"* - 

2 s {i-e 2 )did 2 



D G V G and ii* T Dii* < V* T D Bi i* 



(24) 



(25) 
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The next lemma is central to the proof of our main result. 
Lemma 5. If (R\, R%, d±, cfo) is strict-sense achievable, then 

R l +R 2 > inf max(fl coop (e),i? sum (ff)). (26) 
0e(-l,l) 

Proof. By hypothesis there exists a code {f[ n \ f^\ <fi, V 5 ^"') satisfying (2). 
Then 



= h(y n )-h(y n fi n) (y?),ft\v2)), 



(27) 

where h(-) denotes differential entropy. But the first term on the right-hand 
side satisfies 

%") = | log[(27r e ) 2 |K y |] (28) 
while the second term satisfies 

n 

h (y n \fi n) (ynft\y%)) = E ft (y B (i)|/i (n) (yi),/i n) (»2),y n (i -i-K 

n 



(29) 



since conditioning reduces entropy. Let Di denote the covariance matrix of 

y"W-y™» 



Di = E 



(y"(i)-y"«)(y"«-y"«) J 



(30) 



and let 



n 



denote the error covariance matrix of the code. We may assume that ip\ 
and ip^ are MMSE estimators, in which case Theorem 9.6.5 in Cover and 
Thomas [15] implies that 

h(y n (i)-y n (i))< ilog[(27r e ) 2 |A| 
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Applying the concavity of log-det [15, Theorem 16.8.1], we have 
i 1 n 1 

-h(y n fi n) (y?)jt\y%)) < -£ilog[(2^e) 2 |A 



< 



ilog [(2ne) 2 \b\ 



(31) 
(32) 



Combining this inequality with (27) and (28) gives 

Sl+fl2 >i log+ &l. 



(33) 



Now (27) implies that 

h{y n \A n \ynfi n \v2)) 

Thus D must be nonsingular and hence positive definite. Let us write it as 



> — oo. 



D = 



di 6y d\d 2 
W d\d 2 d 2 



where d\ < di, d 2 < d 2 , and 9 is in (—1,1). Define 



^_ 6\fd\d 2 
\/d\d 2 

and note that <f> is in (—1, 1). Then JD ^ — D is diagonal 



d x - dx 
d 2 - d 2 



Since d\ < d\ and d 2 < d 2 , it follows that — D is positive semidefinite, i.e., 
D < D<j,. In particular, \D\ < \D<p\ [16, Corollary 7.7.4]. This implies that 



i?l+i?2>il0g+^=i?coopW>). 



(34) 



Next observe that 
E 

In particular, we have 



y W - m y W 



71 



m y W-m y («, 



= /x Difi . 



= fj,* T b[A* < n* T D^n* , (35) 
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i.e., this code achieves distortion n* T D^fi* for the /x*-sum problem. Lemma 3 
then implies that 

R!+R 2 >iW(0). 
Combining this with (34) gives 

Rl + R2 > max(i? C oop(0),-Rsum (0))- 

The conclusion follows by taking the infimum over in (—1, 1). □ 

The next step is to evaluate the infimum in (26). Examples of -Rcoop(') 
and -Rsum(') are shown in Fig. 5. We show that these two functions always 
intersect at the correlation coefficient of D* , 9*, and at this point, they equal 
the min-max. 



Lemma 6. 



inf max (i?coop(^)j ^snm(^)) — Rcoop(@*) 

= RsumW ) 



1, \Ky\ 



2 LD 



(l-p 2 )P(d u d 2 ) 



2did 2 

Proof. Let us write D* , the matrix in T>q with diagonal entries (d\, d 2 ), as 



D* 



di 9*^fd^d 2 
T\/d\d 2 d 2 



Then observe that since 9* > 0, if 9 > 9* , we have 

max(i? coop (0), R sum (9)) > R coop (9) > R coop (9*) = ± log SI (36) 
On the other hand, if 9 < 9*, then since i?sum(•) is nonincreasing, 

max(i? coop (0), R sum (9)) > R sunl (9) > R sum (9*) = X - log j^J, (37) 
where we have used the fact that D* solves the /i.*-sum problem. It follows that 

1 ■ \Ky\ 



inf max(i? coop (#), R snm {9)) = R coop (9*) = R sum (9*) = - lo b 

1,1) A \U I 

We conclude the proof by invoking the formula for the determinant of a matrix 
mV G (21). □ 
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Proof of Theorem 1. As discussed in Section 4, it suffices to show that 



K*(di,d2) C^* um (d 1 ,d 2 ) 



Lemmas 5 and 6 together imply that if the rate-distortion vector R 2 , d±, d 2 ) 
is strict-sense achievable and {d\,d 2 ) is in diag(2?c), then 



Ri 



R-2>\ l0g^ 



(l- P 2 )P(d 1 ,d 2 ) 



2d 1 d 2 



(38) 



On the other hand, Lemma 2 implies this inequality if (i?i, R2, di, d 2 ) is strict- 
sense achievable and (d\,d 2 ) in not in diag(2?c). It follows that (38) holds 
whenever (i? 1; R 2 , di, d 2 ) is strict-sense achievable. Since the right-hand side is 
continuous in [di,d 2 ), this implies that if the point (Ri, R 2 , d\, d 2 ) is in 1ZV*, 
then (38) again holds. This implies the desired conclusion. □ 



5.1 Reprise 

The argument used in the converse proof can be summarized as follows. Since 
the distortion constraints only constrain the magnitude of the individual er- 
rors, and not their correlation, we view the determination of the sum-rate as 
an implicit minimization over all possible error covariance matrices, subject to 
upper bounds on the diagonal elements. We then lower bound the sum rate for 
each possible error covariance matrix using two approaches. First, we consider 
the rate needed by a centralized encoder to achieve the given error covariance 
matrix. Second, we use the existing characterization of the rate region for the 
CEO problem to solve the /i-sum problem for some fi vectors. This solution is 
then used to lower bound the sum rate of the problem under study for a given 
error covariance matrix. The first bound is most effective when the correlation 
between the errors is large. The second bound is most effective when the corre- 
lation is small. We therefore form a composite bound by taking the maximum 
of these two lower bounds. The argument is illustrated in Figs. 5 and 6. Note 
that both of the lower bounds are needed. 



6 The M-sums Problem 

Consider next a generalization of the classical problem in which the decoder 
attempts to estimate fijy for a given set of vectors . . . , fij. We may assume 
without loss of generality that these vectors arc distinct and have unit norm. 
Define the matrix 

*f=K ft'" fJ-A 

consisting of the column vectors fi 11 . . . , \x j side-by-side. The problem is then 
to reproduce the vector M T y subject to separate constraints on the average 
squared error of each component. We call this the M-sums problem. Note 
that the classical quadratic Gaussian two-encoder source coding problem can 
be viewed as an instance of the iVf-sums problem with M equal to the identity 
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matrix. We will show that the techniques used to solve that problem can be used 
to solve the general vM-sums problem if the vectors . . . fij satisfy a certain 
condition. Specifically, we will require that the product of the two coordinates 
of each vector is nonnegative 

/iji-Mia>0 Vje{l,...,J}. (39) 

This condition is satisfied if and only for each j either both coordinates of fij are 
nonnegative or both are nonpositive. From a source-coding perspective, these 
two cases are essentially equivalent, so for simplicity we will assume that the 
components of fij are nonnegative for each j. 

The condition in (39) depends on our standing assumption that < p < 1. 
If p is negative, then the condition in (39) becomes 

^■ti j2 <Q Mj &{!,..., J}. (40) 

Note that either way, the condition includes the case when M is the identity 
matrix, i.e., the classical version of the problem. 

6.1 Main Result 

In this section, we use lZ*(dj) to denote the rate region of the /j^-sum problem 
with distortion constraint dj. Let R* um (di, . . . ,dj) denote the minimum sum 
rate for the Af-sums problem achieved by the separation-based scheme 

-Rsum^l, ■ • -,dj) 

= inf | i log ip^j! : D G V G and fijD^ < dj Vj e {1, . . . , J} | . (41) 

Then let lZ* um (di, .. . ,dj) denote the set of rate pairs whose sum is at least 
Rtum(di, ■ ■ ■ ,dj) 

^ um (di, . . . , dj) = {(Ri,R 2 ) :R 1 + R 2 > R* sum (di, dj)}. (42) 

In terms of these sets, the separation-based architecture achieves the following 
inner bound. 

Lemma 7. For the M-sums problem, the separation-based architecture achieves 
the rates 

J 

n* sum (d 1 ,...,d J )nf]n*(d J ). 

i=i 

The proof is elementary but somewhat involved and is given in Appendix E. 
The main result of this section is the following theorem that shows that this 
inner bound equals the rate region. 

Theorem 2. The rate region of the M-sums problem equals 

J 

n: um (d 1 ,...,d J )nf)K*(d J ). (43) 

i=i 
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The proof parallels that of Theorem 1. In particular, we use functions similar 
to -RcoopG) and i? S um( - )- The details are given in Appendix F. 

By mimicking the proof of Proposition 2, one can show that the separation- 
based architecture achieves the rates in (43) even if the source is not Gaussian. 
Theorem 2 then implies that, as with the classical version of the problem, the 
separation-based inner bound and the rate region are both smallest for a Gaus- 
sian source. 



6.2 The Remote-source Problem 

As an application of Theorem 2, consider the remote-source version of the orig- 
inal problem. Here the encoders' observations are viewed as an underlying 
source, y, plus additive noise 



yi 

02 



Vi 

m 



m 

n 2 



where y, ri\, and n 2 are independent and Gaussian. We assume these random 
variables have zero mean and the following second moments 



E[yl\ 

mi] 

E [my 2 ] 
E[nl] 



<j{ < 1 
of < 1 
P 

1-a 2 



E\ni] = 1 - ai 



so that y has covariance matrix K y . The aim is to reproduce y\ and y 2 subject 
to distortion constraints d± and d 2l respectively. A partial characterization of 
the rate region for this problem was obtained by Oohama [17]. By coupling this 
problem to an iVf-sums problem, we can determine the rate region completely. 

Corollary 1. The rate region for the remote- source problem with distortion con- 
straints d\ and d 2 equals the rate region for the M-sums problem with distortion 
constraints d\ — 71 and d 2 — 72, where 



M 



1 



4-P 2 



and 



71 



72 



1 -p 2 
(a 2 2 -p 2 )(l~4) 
I-P 2 



\Mi M2] 



Proof. Standard calculations show that -E[yi|y] = I^Jy, and in particular, y\ 
can be written 



Vi 



T 

MiY 
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where hi is Gaussian, independent of y, and has mean zero and variance 71. 
Then for any random variable y such that y *-> y <-» y, we have 

E[(m - vif] = E[(m - Mi y + MiV - vi) 2 } 

= E[(fi 1 +tfy-y 1 ) 2 ] 

= ll +E[(rfy-y 1 ) 2 }. (44) 
Now consider a pair of encoders, f[ n ^ and , and a decoder (ip^\ V^) with 

5? = ^ n) (/i (n) (yr),/i n) (yJ)) i = i,2. 

Since the source is i.i.d., for any time i, we have 
Thus by (44) , it follows that 

E[{yUi)-m)) 2 ] =7i + ^y"W-yi«) 2 ]. (45) 

By averaging both sides of this equation over time, we see that any code that 
achieves distortion d\ for yi must achieve distortion d\ — 71 for fify and vice 
versa. Likewise, any code that achieves distortion di for 7/2 must achieve distor- 
tion d2 — 72 for fJ^y and vice versa. The conclusion follows. □ 

Observe that this proof does not require the assumption that < p < 1, only 
that p 2 < 1. If < /> < 1, then and /x 2 satisfy the condition in (39). On the 
other hand, if — 1 < p < 0, then fi 1 and fi 2 satisfy the condition in (40). Since 
the cases p = 0, p = — 1, and p = 1 can be solved using existing techniques, the 
rate region for the remote source problem is solved for any value of p. 



7 Many Sources 



Our technique can be used to determine the sum rate for more than two sources 
if a certain symmetry condition holds. Suppose now that there arc L jointly 
Gaussian sources, j/i, . . . , j/l, with covariance matrix 



K, 



1 P 
P 1 

P P 



for some < p < 1. That is, the source components arc Gaussian, exchangeable, 
and positively correlated. We assume that the sources are separately encoded, as 
shown in Fig. 9, and that L distortion constraints are imposed on the individual 
reproductions 

n 

<*<>-y>[(tf?(i)-tf?(i)) 2 ] foralHin{l,...,L}. 
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The separation-based scheme yields an inner bound on the rate region, and in 
particular, an upper bound on the sum rate. As in the case of two sources, let 
T>g denote the set of matrices D such that 

D 1 = Ky 1 + A 

for some diagonal and positive semidefinite matrix A. The sum rate achieved 
by the separation-based scheme is then 

irfj^log^ : D G T>g and ej De e < d t V t € {1, . . . , L}| , (46) 

where eg denotes the vector with one in position £ and zero elsewhere. By 
following the proof of Theorem 1, one can show that this sum rate is optimal if 
the distortion constraints d\, . . . , dj, are equal. 

Theorem 3. If d\ = d,2 = ■ ■ ■ = cLl = d, then the separation-based architecture 
is sum-rate optimal. In particular, the sum rate is given by (46). Furthermore, 
in this case the infimum in (46) is achieved by a D in T>q of the form 

D 1 = Ky 1 + XI, 

for some A > 0. 

The proof is given in Appendix G. As with the A^f-sums problem, it is 
possible to mimic Proposition 2 and show that the separation-based architecture 
achieves the sum rate in (46) even if the source is not Gaussian. It follows 
that the Gaussian source has the largest sum rate among all exchangeable and 
positively correlated sources when all of the distortion constraints are equal. 

8 Concluding Remarks 

We determined the rate region of the quadratic Gaussian two-encoder source- 
coding problem. This result implies that a simple architecture that separates 
the analog and digital aspects of the compression is optimal, and that this 
architecture requires higher rates to send a Gaussian source than it does to send 
any other source with the same covariance. We also described how our proof 
technique can be extended to determine the sum rate of some generalizations of 
this problem. We now comment on two aspects of our results. 

8.1 An Extremal Result 

One consequence of our main result is that there is no loss of optimality in 
using Gaussian auxiliary random variables in the separation-based inner bound. 
More precisely, the regions TZ l {d\, c?2) and TZo(di, (fe) defined in (5) and (6) are 
equal. In particular, these two regions have the same sum rate. Thus to the 
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optimization problem 



minimize 



/(y;u) 



subject to 



Ui <-> 2/1 <-> 2/2 <-> "2 



(47) 



S^-E^u]) 2 ]^ j <G {1, 2} 



we can add the constraint 



(y, u) is jointly Gaussian 



without changing the optimal value. The same is true, of course, of the opti- 
mization problem 



This is akin to the well-known fact that the Gaussian distribution maximizes 
entropy for a given covariance. But this result is more subtle in that the con- 
ditional covariance of y given u is not fixed, and by using non-Gaussian u, one 
can potentially realize conditional covariances that are unattainable with Gaus- 
sian distributions. Evidently the entropy-maximizing property of the Gaussian 
distribution more than compensates for its smaller set of achievable conditional 
covariances. 

Using Theorem 2, it is possible to generalize this result to allow distortion 
constraints on linear combinations of the source variables j/i and 2/2- It is also 
possible to prove a multi-letter version of this result by first proving a multi- 
letter version of the inner bound, in which several source symbols are treated as 
a single "supersymbol." Whether one can prove any of these extremal results 
without reference to the source-coding setup that is the subject of this paper is 
an interesting open question. 

8.2 Source Augmentation 

The most noteworthy aspect of our proof is the random variable x that we 
add to the source y in Appendix C to solve the /x-sum problem. Unlike other 
more typical auxiliary random variables, x does not represent a component 
of the code. Rather, it is used to aid the analysis by inducing conditional 
independence among the observations, which allows us to couple our problem 
to a CEO problem. Of course, there are many random variables that will induce 
conditional independence. The role of Lemma 4 is to identify the best one. 

This technique of augmenting the source to induce conditional independence 
has proven useful in other contexts as well. Ozarow [18] used it to prove the 
converse for the Gaussian two-descriptions problem. Wang and Viswanath [19] 
used it to determine the sum rate for the Gaussian vector multiple-descriptions 
problem with individual and central decoders. Wagner and Anantharam [11, 12] 



maximize ft,(y|u) 

subject to ui <-> 2/1 <-> 2/2 <-> U2 



(48) 



E[( yj - E[ yj \u}) 2 ] <d 3 j €{1,2}. 
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used it to prove an outer bound for the discrete multiterminal source-coding 
problem. 

Recently, we have generalized the CEO result to sources whose correlation 
satisfies a certain tree condition [20] (see also [21]). This suggests an approach 
for generalizing the results in this paper. Specifically, one could potentially 
augment the source to couple a given distributed source coding problem to this 
tree problem instead of the more restrictive CEO problem. Determining whether 
this revised approach yields stronger results is a worthwhile question for future 
research. 
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A Proof of Proposition 2 

Let (Ri,R 2 ) be a rate pair in 

n\{di) n n* 2 {d 2 ) n n* sum (d 1 ,d 2 ). 

By Lemma 1, there exists a u in UG{di,d 2 ) such that 

Ri > ^(z/i;wiK) 

i?2 > /(lft;u2|«i) (49) 

Ri + i? 2 > /(y;u). 

Now u can be expressed as 

u\ = ciyi + zi 
u 2 = C2V2 + z 2 

for some coefficients c\ and c 2 in [0, 1), where z±, z 2 , and y are independent and 
z is Gaussian. Now construct auxiliary random variables u\ and u 2 for the true 
source via 

iii = cij/i + zi 
u 2 =c 2 y 2 + z 2 

with z independent of y. Note that the Markov condition 

Ul <-> Vl <-> J/2 <-* U2 

is satisfied and that (y, u) and (y, u) have the same second-order statistics. 
Thus the error in the linear minimum mean-squared error estimate of yj given 
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u, LMMSE(yjl'Q), equals the error in the linear minimum mean-squared error 
estimate of yj given u, 

LMMSE(&|u) = LMMSE( % |u) j e {1, 2}. (50) 

But conditional expectation minimizes mean square error, 

E[{y 3 - E[ Vj \u]) 2 ] < LMMSEfe|u) j G {1,2}, (51) 

and for jointly Gaussian random variables, the linear minimum mean-squared 
error estimate is also the conditional expectation [22, Theorems 9.1-1 and 9.1-2]. 
Since u is in Uc(di, d 2 ), this implies 

£[fe--i%|u]) 2 ] <dj je{i,2}. 

It follows that u is in U(di,d 2 ). Next, we show that (R\ 1 R 2 ) satisfies 

Ri > i(yi;ui\u 2 ) 

R 2 >i{y 2 \u 2 \u{) (52) 
R 1 + R 2 >I(y;u). 

To prove this, it suffices to show that 

l(yi;ui\u 2 ) > I(yi;&i\u2) 
I(V2;u 2 \u 1 ) > I(y 2 ;u 2 \ui) 

7(y;u) >/(y;u). (53) 

By symmetry, it suffices to prove the last two inequalities. Let 

E[uiU 2 ] 

a2 = -bRT = E[U1U2] - 

Then we have 

I{y 2 ;u2\ui) ( = h(u 2 \ui) - h{u 2 \y 2 ) 

= h(u 2 - a 2 u\\u\) - h(z 2 ) 

(c) 

< h(v,2 - a 2 ui) - h(z 2 ) 
(d) 

< h(u 2 - a 2 ui) - h{z 2 ) 
= I(y2;u 2 \u 1 ), 

where 

(a) follows from the Markov condition u 2 <-> y 2 <-> Mi, 

(6) follows because differential entropy is invariant to shifts, 
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(c) follows because conditioning reduces differential entropy, 

(d) follows from the fact that u has the same covariance as u, and the Gaussian 
distribution maximizes differential entropy for a given variance, and 

(e) follows because steps (c) and (d) are tight if (y, u) is Gaussian. 
We can prove (53) via similar reasoning 



(54) 



I(y;u) = h(u)-h(u\y) 
= h(u) - h(z) 
< h(u) - h(z) 
= /(y;u). 

It follows that the rate pair (i?i, R 2 ) belongs to TZ l (di, d 2 ). 

B Converse for a Special Case 

Proof of Lemma 2. The conclusion is easily verified if mm(di,d2) > 1, so as- 
sume instead that min(di,c?2) < 1- Without loss of generality, let us assume 
that di = min(di, c^)- Then by (14), we must have 

p 2 d 1 + 1 - p 2 < d 2 . 

If this holds, then 

Kt nm (d 1 ,p 2 d 1 + 1 - p 2 ) C K* sum {d u d 2 ). 
But one can verify directly that 

K* sum (d 1 ,p 2 d 1 + 1- p 2 ) = |(i?i,i? 2 ) .R1+R2 > \ l °S^Y (55) 

Now via calculus one can show that if (i?i,i?2) is in lZ*(di) then i?i and R 2 
must satisfy 

Ri + R2 > -logj. 

It follows that 

K\(di) C TZ* um (d 1 ,d 2 ). 

In particular, we have 

ni(d 1 )nni(d 2 )=nUdi)nn t 2 (d 2 )nn* um (d 1 ,d 2 ). (56) 

The result then follows from (11) and Lemma 1. □ 
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C The /x-sum Problem 



In this appendix, we determine the rate region for the /x-sum problem if fi 
satisfies [i\ ■ [12 > 0. If fii ■ /i 2 = 0, then the rate region has already been 
determined by Oohama [5], so we shall assume that /^i • [1% > 0. 

We begin by noting that if fi and the allowable distortion are both scaled 
by the same factor, then the rate region remains unchanged. We may therefore 
assume that fi is normalized. In particular, we may assume that 



fii-fi2 = — - SNRx • SNR 2 

P 



where 



and 



SNRi 
SNR 2 

7- 1 



Mi , 
2 \ —+P 



1 - P 2 \P2 
P f P2 



1 + SNRi +SNR 2 



(57) 

(58) 
(59) 

(60) 



This normalization is convenient because, as we shall see, it admits a particularly 
simple coupling to a CEO problem. 

Lemma 8. Suppose the vector fj, satisfies fii-fJ-2 > and the normalization (57). 
Then the rate region for the fi-sum problem with allowable distortion d equals 



(i?i, R2) : there exist r^ > 0, r 2 > such that 



Ri > 2 l0 ? 



i?2 > - loe 



R 1 + R 2 > - log H 



1 


d 


+ 


1 




f 




d 


+ 


1 




f 




d 


+ 


1 



(l + SNR 2 (l-2- 2r2 ))- 1 
(l + SNR 1 (l-2- 2ri ))- 1 
+ r\ + r 2 



— < 1 + VSNR 7 (1 -2- 2r ol. 

+7_ k I 



In particular, the sum rate equals 



inf < 


jlog + 


1 






d + l. 



n 



'2 



+ ri + r 2 : r\ > 0, r 2 > 0, and 



l + ESNR^l^-^-L-l 
j=i ) 



(61) 



(62) 
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or, equivalently, 

inf | i log i^jl :DeV G and ^Dfi < d J . (63) 

Furthermore, the infimum in (63) is achieved by a unique feasible D. 
Proof. Let 

/ <;mr. \ 1/2 

je{l,2}. (64) 



SNR 3 \ 1/2 
1 + SNRjJ 

Clearly ai < 1 and 02 < 1. Using (58) and (59), one can verify that aia 2 = p. 
It follows that a\ and a 2 are each contained in (p, 1). Let x, n-i, and 712 be 
independent zero-mean Gaussian random variables with variances 

E[x 2 } = 1 

E[nj] = l-aj ie{l,2}. 

Since (aix + rii, a^x + n.2) has covariance matrix we can couple these vari- 
ables to y to create a CEO problem 



y\ = a\x + n\ 
y 2 = a 2 x + n 2 . 

The SNR notation is justified by the fact that 

SNR — — — Var(ajx) 

bNKj " 1-a 2 - V ar( % ) 

Now starting with (57), we have 



(65) 



Mi ■ M2 = VmTT^ • -7= • VSNRiSNR 2 

Substituting for SNRi and SNR 2 and rearranging gives 



/ PMl + P A*2 + PMl 

Mi=7'W : 7j — ■ (66) 



But observe that 

^ +PfH =1 + SNR 1= 1 



(l-/>> 2 1-0? 

and 



V M2 + Pll\ 

Substituting these equations into (66) gives 

1 



01. 



pi = 7 • ai • 



1-a 2 



7 L - (67) 



01 
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Similarly, 



= 7 ' 



SNR 2 



a 2 



(68) 



Now using the fact that axa 2 = p, we have 



1 



[oi a 2 ] ■ K = -[a x - pa 2 a 2 - pa{\ 

" 1 — or 



P~ 
1 



1 — a\a\ 



n 9 9 9 9 9 

ai - afa 2 a A 2 — a\a 2 



a i 



(69) 



Substituting for a\ and a 2 using (64), this gives 

"7SNR1 7SNR2 



[ai a 2 ] ■ Ky 1 = 



Ox 



2 



Thus we have 



It follows that 



E\x\y] = 7 ( SNRx^i + SNR,^ 
ax a 2 



(70) 



E[x\y] = M T y, 
and in particular, x can be written 

x = fJ> T y + n, 

where h is Gaussian, independent of y, and has variance 



I -[a x a 2 ]-K~ 1 - 



ax 
a 2 



= 1 - 



= 7- 



7SNR1 7SNR2 



"1 



« 2 



ax 
a 2 



(71) 



Then for any random variable u such that x <-> y ^ u, by a calculation similar 
to (44), we have 



E[(x - E[x[u]f] = 7 + E[(n T y ~ E[fi T y\u}) 2 }. 



(72) 



As in the proof of Corollary 1, it follows that any code that achieves distortion d 
for the /j-sum problem must achieve distortion d + 7 for the CEO problem (65) 
and vice versa. The characterization of the rate region in (61) and the sum rate 
in (62) now follow from existing results on the CEO problem [8, 10]. To show 
that (63) equals (62), we first show that (62) can be rewritten as 



inf J i log 



l + ^SNR J (l-2" 2 ^) 



j'=i 



r\ + r 2 : r\ > 0, r 2 > 0, and 



l+£sNR^l-2-^)> * }. 
j=i ) 



(73) 
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To see this, note that the two optimization problems differ only in the objective, 
and both objectives are increasing functions of r± and r^. Now if d > 1 —7, then 
both infima are zero. On the other hand, if d < 1 — 7, then in both problems, we 
may assume without loss of generality that the constraint is met with equality 



l + X)SNR J -(l-2- 



2f ,- 



1 



rf + 7 



(74) 



But if the constraint is met with equality, then the two objectives are equal. 
Thus the two optimization problems are equivalent. 

Let u be a distributed Gaussian test channel such that 

x «-> y «-> u. 

If we define 

fj = I(yj]Uj\x) j €{1,2}, 
then a standard calculation shows that 



E[(x - E[x\u]Y x 



(75) 



Thus the expression in (73) equals 

inf |/(x;u) + I(yi; Ui\x) + I(y2',U2\x) : (x, y, u) are jointly Gaussian, 



x <-> y u 

u\ <-> ?/i *-> 2/2 <-> W2 



(76) 



- £[x|u]) 2 ] < d + 7 }- 
Now since (j/i, Hi) (j/2, ^2) and x «-> y «-> u, we have 

7(x;u) +/(yi;ui|ar) + /(y 2 ; U2I2:) = J(a:;u) + J(y;u|x) 

= /(y;u). 

Applying (72) again, we can write the infimum in (76) as 

inf |/(y; u) : (y, u) are jointly Gaussian, 
ui <-> y\ <-> y 2 <-> "2 



(77) 



which equals 



SK^y-E^ylu]) 2 ]^^, 
inf I i log H : -D € T>q and /x T Z)/x < d j . (78) 
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Now since the quantity 

2 
3=1 

is strictly concave, it follows that the infimum in (62) is achieved by a unique 
feasible point. This in turn implies that the infimum in (73) is achieved by a 
unique feasible point. By the equivalence between the feasible points in (73) 
and (78) it follows that the infimum in (78) is also achieved by a unique feasible 
point. □ 



D Every D* Solves a /x-sum Problem 



Proof of Lemma 4- Without loss of generality, we may assume that fi* has been 
scaled so that it satisfies the normalization (57). Then the sum rate for the //,*- 
sum problem with allowable distortion 



is given by 





1 




d * +7. 


1 + 


ESNR 

3=1 



fi D fi 



7*1 + r2 : T\ > 0, T2 > 0, and 

. __o_^ . 1 1 



(79) 



1 



where SNRi, SNR2, and 7 were defined in equations (58) through (60). The 
remainder of the proof consists of three parts: 

(I) We identify candidate optimizers for (79), r\ and r|, in terms of D* . 

(II) We show that r\ and r\ achieve the infimum in (79). 

(Ill) We show that at r* and r|, the objective 

1 



\ log 



equals 



d* +7 

ilog^ 
2 S ID* I 



+ rT + r; 



Part /. Since D* is in Dg, there exists A* > and A*> > 0, such that 



D* 



k: 



A* 

Xn 



(80) 
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Our candidate optimizers are then 
. 1 



log 1 



A* 



1 + SNR, 



ie{i,2}. 



(81) 



This formula can be understood as follows. Since D* is in T>c, there exists a 
distributed Gaussian test channel u* such that Cov(y|u*) = D* . Now u* can 
be written 



A* 



1 + A* 



y-j+zj j€{l,2} 



where z is an independent Gaussian vector with covariance matrix 



(1 + At)" 1 

o (l + A^r 1 



As in the previous appendix, let 



SNR, 



1 + SNR, 



1/2 



J'e{i,2} 



(82) 



and let x, m, and n 2 be zero mean Gaussian random variables with 

E[x 2 ] = 1 

E[n]\ = 1 - a] je{l,2}. 

Then couple these variables to (y, u*) such that 

j/l = ayx + Tlx 
J/2 = a 2 x + n 2 

and x «-* y <-» u*. It then follows that 

^ = ^og(i + A;(i- a ?)) 

= /(j/j;uj|a;). 

Part II. Next we show that r* and solve the optimization problem (79). 
Since the optimization problem is convex, it suffices to show that r* and r\ 
satisfy the Karush-Kuhn- Tucker (KKT) conditions [23, Section 5.5.3]. The La- 
grangian for this optimization problem is 



L(ri,r 2 ,v) 



1 



log 



d* +7 



+ r\ + r 2 - v 



l + ^SNR 3 -(l-2" 



2r, 



rf* + 7 



(83) 
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Thus it suffices to show that 

2 2, 1 = i/'SNRj Vje{l,2} 
l + ]TSNR,(l-2-^) = -l 



d* +7 



for some > 0. To show (84), note that (80) implies that 



l + (l- P 2 )Xl 
-P 



l + (l-p 2 )A 2 



Then define 



Since, by definition, 



3 * = 1 + (1 - p 2 )A* JG{1,2}. 



D* 



0*^d^ d 2 



it follows from (86) that 



<h _ £2 
d 2 si ' 

Referring to the definition of SNRi and SNR2, this implies 



SNRi = 
SNR 2 = 



1 -p 2 
P 

1-p 2 



^+P 



Combining (81) and (87), we have 



(l-p 2 )(l + SNRj)(2 2 'i -1) + 1. 



Thus (84) is equivalent to 



1 = i/*SNRj- Vje{l,2} 



(84) 
(85) 



(86) 
(87) 



(88) 
(89) 



(90) 



(l-p2)(H-SNRj) 

But by using (88) and (89), one can verify that this pair of conditions holds if 

„2 



1-p* 



sls 2 +PVsfsJ 



p P (st + s* 2 ) + ^44(1 + p 2 )' 



(91) 
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This establishes (90) and hence (84). Now as in the previous appendix we have 

-l 



E[(x - E[x\u*}f 



-2r' 



l + 5^SNR,-(l-2- 
= 7 + £[(M* T y-£[M* T y|u*]) 2 ] 



This establishes (85) and the optimality of r* and rjj. 
Part III. It only remains to show that 



1 



log 



1 



■7. 



r 2 ~ 2 log 



Observe that the left-hand side equals 



(92) 



(93) 



i=i 



Repeating the argument in (77), we have 



I(x; u*) + £ J( % ; Ui *|z) = 7(y; u*) = 1 log |g| 



j'=i 



□ 



E Achievability for iV^-sums 

Before proving Lemma 7, we examine some of the properties of the constituent 
regions TZj(dj). Without loss of generality, we focus on TZ*(di). 
Let hi(-) denote the function whose epigraph is TZ*(di) 

n* 1 {d 1 ) = {(R 1 ,R 2 ):R 2 >h 1 (R 1 )}, (94) 

which may equal infinity for some R\. Note that h\{-) is nonincreasing. Since 
TZ\{d\) is closed and convex, /ii(-) must be continuous on its effective domain 
[24, Theorems 7.1 and 10.1]. Thus h\{-) is closed and proper [24]. 

For i?i in the effective domain of R\, let dhi{R\) denote the sub differential 
of hi(-) at i?i 

dhi(Ri) = {s : hi(R) > hi(Ri) + s(R - R t ) V R}. (95) 

Note that dhi(Ri) can be interpreted geometrically as the set of slopes of all 
supporting lines at i?i . From standard results in convex analysis [24, Section 24] , 
dhi(Ri) is the interval between the left and right derivatives of hi(-) at Ri. The 
graph of dhi(-), 

{(R, s) : se dhi(R)} 
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w 


= {Ri : 


X 


= {Ri : 


Y 


= {Ri ■ 


z 


= {Rl : 



resembles the graph of a nondecreasing function, except that any jump disconti- 
nuities have been "filled in" with vertical segments. As such, dhi(-) is monotonic 
in the sense that if i?i < Ri then s < s for any s in dhi(Ri) and s in dhi(Ri). 
We now partition the effective domain of hi(-) into three parts. Let 

max(dhi(Ri)) < -1} 
min(9/n(iJi)) < -1 < max(dhi(Ri))} 
-1 < min(dhx(Ri)) < 0} 

dhi(Rx) = {0}}. (96) 

These intervals are depicted in Fig. 7. We call W the steep part of 72-*(<ii) and 
Y and Z the shallow part. Some of these intervals might be empty in some 
cases. Note that the sum rate decreases as one moves left-to-right in W and 
increases as one moves left-to-right in Y and Z. 

Next we associate each point i?i in the effective domain of hi(-) with a test 
channel that meets the fi 1 distortion constraint with equality. Since (i?i, hi(Ri)) 
is on the boundary of TZ*(di), it must be on the boundary of the contrapoly- 
matroid of some test channel D satisfying fij Dfi 1 = d\. Suppose first that R\ 
is in W. Then TZ*(di) has a supporting line with slope s < — 1 at (i?i, 
Since the contrapolymatroid associated with D is contained in TZ*(di), this 
contrapolymatroid must also be supported by a line with slope s < —1 at 
(Ri,hi(Ri)). This implies that (Ri, is on the vertical portion of the 

boundary of this contrapolymatroid. 

In fact, by the definition of /ii(-), (Ri, hi(Ri)) must be the left corner point 
of the contrapolymatroid of this test channel. We then associate R\ with the 
unique test channel whose left corner point is (Ri, hi(Ri)). Likewise, to ev- 
ery R\ in Y, we associate the unique test channel whose right corner point is 
(RuhiiRt)). 

If Z is nonempty then X U Y is bounded. In this Ri — > sup X UY, 

the associated test channels will converge to a test channel D. We associate 
this test channel with all Ri in Z . 

If Ri is in X, then TZ\{d\) is supported at (i?i, h%(Ri)) by a line with slope 
— 1. This implies that {R\,hi(Ri}) is sum-rate optimal. We then associate all 
Ri in X with the unique test channel that is sum-rate optimal (sec Lemma 8). 

Note that the end-points of the interval X must correspond to the corner 
points of the sum-rate optimal contrapolymatroid. It follows that the associated 
test-channels vary continuously with R\ over the entire effective domain of hi(-). 
The test channels also vary monotonically in the sense that if R\ < R\ and A 
and A are the associated test channels, then 



A = A 



-Ai 
A 2 



for some nonnegative numbers Ai and A 2 . Note that for each test channel that 
meets the distortion constraint with equality, at least one of its corner points 
must be on the boundary of JZ*(di). 
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Now consider a second vector, fj, 2 , an( i suppose that \i 2 weights more 
heavily than n 1 does 

^ > a™. (97) 

M21 Mil 

Next we show that as one moves left-to- right along the boundary of 71* (di), the 
distortion that the associated test channels induce on fi 2 y is nondecreasing. 

Lemma 9. Suppose R\ < R±, and let D and D be the associated test channels. 
Then 



Proof. Define A and A by 



d 1 = k: 



d = k: 



Since R\ < i?i , we know that 

A = A 



-Ai 
A 2 



for some nonncgative numbers Ai and X 2 . Then it follows that 



D 



Furthermore, we know that 



b 1 



-Ai 
A 2 



fiiDfi 1 = fi^bfi 1 = di. 
To establish (98), we will show that 

< [i\b[i 2 [i\D[i x n T \ Dyb r vlDn2 



(98) 

(99) 
(100) 



(101) 



Now D can be written 



b 1 



a —b 
-b c 



(102) 



where a, b, and c are nonnegative. In terms of these parameters, we have 
1 



D 
D 



,c — b 2 



c b 
b a 

1 



(a- \ 1 )(c + X 2 )-P 



c + \ 2 b 
b a — Ai 



A tedious but straightforward calculation shows that the expression on the right- 
hand side of (101) equals 

(Aic + aA 2 )(/ii 1 ^22 ~ M12M21) + 2fo(AiMi2M22 + A2^ii^2i)(/iii/ i 22 - M12M21) 
(ac-6 2 )((a-Ai)(c + A 2 )-6 2 ) 
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which is nonnegative because \D\ > 0, \D\ > 0, and 

M11M22 - M12M21 > 

by (97). □ 

From the proof, we can see that if the inequality in (97) is reversed, then 
the inequality in (98) is also reversed. 
We now turn to the proof that 

J 

Tz: um (d 1 ,...,dj)nf]n*(d,) (103) 

3=1 

is achievable. Recall that TZ* um (di, . . . dj) is defined as 

^suml^b •■■>dj) = {(iZl, R%) ■ Rl + R2 > Rtum(dl, ■ ■ ■ , dj)} 

where 

#sum(^li ■ ■ ■ i d j) 

= inf U log ^ : D G V G and /xJZ)^- • r/, 7 • {1 J}| . 

We begin by showing that this infimum is achieved. 
Lemma 10. There exists a D* in T>q such that 

i^D"n j <d j Vje{i,...,J} 

and 

\ log w\ = inf { 5 log Tor : 19 G Pg and MjT£> ^ - ^' Vi e {1 ' ' ' ' ' ' 

Proof. Let D by any matrix in T>q that satisfies the distortion constraints. The 
infimum is then upper bounded by 

2 8 \D\ 

Now the set of D £ T>c such that 

/<{/}/'. VjG{l,...,J} (104) 

^g^aiog^, (105) 
2 6 |D| - 2 6 |D| 

is compact, and the objective is continuous, so the infimum is achieved. □ 
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Proof of Lemma 7. Recall we are assuming that the vectors fi 1 ,...,fi J are dis- 
tinct and have norm one. We may also assume without loss of generality that 
they have been ordered so that their first coordinates are decreasing 

Mn > M21 > • ■ ■ > fi>n 
which implies that their second coordinates are increasing 

Ml2 < M22 < ' ' ' < M./2- 

Let 

h j (R 1 )=m£{(R 2 :(R 1 ,R 2 )eK* j (d j )} je{l,...,J} (106) 

and 

h sum (Ri) = inf { (R 2 :(R 1 ,R 2 )£ U* sum (d x , d 2 , . . . , dj ) } (107) 

denote the functions whose epigraphs are the constituent regions. The corre- 
sponding function for the intersection is given by 

h(Rx) = max(/ii(i?i), . . . , h J (R 1 ),h sum (R 1 )). 

To show that the intersection is achievable, is suffices to show that for each Ri 
for which h(Ri) < 00, (Ri,hj(Ri)) is achievable for some j or (R 1: h BUm (Ri)) 
is achievable. 

By Lemma 10, there exists a D* in T>q that is sum-rate optimal within this 
class 

// ' D // ,/, :j , {1 7} (108) 

\ bg W\ = ^ G lD[ : D £ Vg and ~ dj Vj G U ' • ■ ' ' J} } • 

(109) 

Let A denote the set of constraints that are active at D* 

A = {] ■ vjD*Hj = dj}, 

which must be nonempty. Let i denote the largest element of A. Thus the \x i 
sum is the one that weights y 2 most heavily of all of the sums whose constraints 
are active at £)*. 

Consider the contrapolymatroid region associated with D* . We shall show 
that the right corner point of this region is on the boundary of IZ^di). Since D* 
achieves the \i i constraint with equality, either the left corner point or the right 
corner point (or both) of its contrapolymatroid region must lie on the boundary 
of lZ*(di). Suppose it is only the left corner point that lies on the boundary 
of 1Z*(di). This corner point would then have to lie on the steep part of the 
boundary (W) of H*(di). Consider moving a small amount to the right along 
the boundary of lZ*(di). Since the distortion for /x • varies continuously as we 
move along the boundary of lZ*(di) for each j, for a sufficiently small movement, 
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none of the distortion constraints in A c will be violated. At the same time, by 
Lemma 9, none of the distortion constraints in A will be violated either. 

Thus we can strictly reduce the sum rate without violating any of the distor- 
tion constraints. This contradicts the assumption that D* is sum-rate optimal 
within the class T>q. Hence it must be the right corner point, (R^R^), of the 
contrapolymatroid that is on the boundary of lZ*(di), as shown in Fig. 8. 

Next consider starting at this corner point and moving to the right along 
the boundary of lZ*(di). By Lemma 9, the fij constraint will remain satisfied 
for all j < i. If, as we move to the right, the /i fe constraint is never satisfied 
with equality for all k > i, then it follows that the shallow portion of ll*(di) to 
the right of R%) 1S achievable. 

If the fi k distortion constraint becomes active for some k > i, then there ex- 
ists a point (Ri,R,2) on the boundary of lZ*(di) whose associated test channel D 
meets both the /Xj and fj, k distortion constraints with equality 

/if = ck (110) 

n T k bn k = d k . (in) 

Since (i?i,i?2) is on the shallow portion of the boundary of lZ*(di), (i?i,i?2) 
must be the right corner point of the contrapolymatroid region associated with 
D. Now at least one of the corner points of D must be on the boundary of 
TZ k (d k ). By Lemma 9, the boundary of 7Z k (d k ) must be below that of 1Z*(di) 
to the left of R\. It follows that it must be the right corner point of D, i.e., 
(Ri, R2), that is on the boundary of TZ k (d k ). We then move to the right along the 
boundary of lZ k (d k ), repeating the above process as necessary as new constraints 
become active. This shows that for each i?i > (Ri,hj(Ri)) is achievable 
for some j. 

An analogous procedure can be followed starting with the left corner point 
of the contrapolymatroid associated with D* . Finally, between these two corner 
points, (Ri, /isum(-Ri)) is achievable. □ 



F Converse for M-sums 



We show in this appendix that the rate region for the Ai'-sums problem is 
contained in 

J 

Kum(di,---,dj)nf]TZ*(d 3 ). 

i=i 

Since the rate region is clearly contained in (~}j =1 lZ*(dj), we only need to show 
that it is contained in lZ* unl (di, . . . , dj). That is, we must show that the sum 
rate is lower bounded by 

Kum(di, ■ ■ ■ ,dj) 

= inf [ \ log i^ji : D e V G and fifD^ < dj Vj e {1, . . . , J}} . (112) 



35 



By Lemma 10, this infimum is achieved by some D* in T>q. If D* = K y , then 
R* um (di, . . . , dj) = and our task is trivial. Suppose therefore that D* ^ K y . 
Then D* must meet at least one of the distortion constraints with equality. 
If D* meets exactly one distortion constraint with equality, then the converse 
is relatively simple, because in this case lZ* um (di, . . . , dj) contains lZ*(dj) for 
some j. 

Lemma 11. Suppose there exists D* in T)q that is sum-rate optimal within 
this class and meets exactly one of the distortion constraints with equality, i.e., 
/J.J D* fi i = dj and fij D*fij < dj for all j ^ i. Then the sum rate for the 
M-sums problem is 

^og^-=Rt um {di,...,dj). 

Proof. It suffices to show that D* is sum-rate optimal for the /i r sum problem 
with distortion di . Recall that one of the corner points of D* lies on the bound- 
ary of the ^-sum rate region. If this point is not sum-rate optimal, then it is 
possible to move a small distance along the boundary of lZ*(di) and strictly de- 
crease the sum rate. Since the distortion associated with fij varies continuously 
as we move along this boundary for each j, it follows that a sufficiently small 
movement will not violate any of the distortion constraints. This contradicts the 
assumption that D* is sum-rate optimal for the Af-sums problem within the 
class T>q. It follows that D* is sum-rate optimal for the /J r sum problem. □ 

It also happens that lZ* um (di, . . . ,dj) contains lZ*(dj) for some j when there 
is an optimizing D* on the "boundary" of T>q. Let T)° G denote the set of D in 
Vq such that 

D 1 = K- 1 + A 

for some diagonal and positive definite A. Let &D = T>q — T)° G , which is the set 
of those D € T>c such that D 1 1 — K ~ 1 is singular. 

Lemma 12. Suppose there exists a D* ^ K y in dT>Q that is sum-rate optimal 
within the class T>q. Then the sum rate for the M-sums problem is given by 

\ XOg ]D^\ = Kim ( d l>" ■><*■/)■ 

Proof. As in the previous proof, it suffices to show that for some i, D* is sum- 
rate optimal for the /i r sum problem with distortion di. Now D* can be written 

D* = (K- 1 +A)-\ (113) 

where A is diagonal, positive scmidefinite, singular, and nonzero. Without loss 
of generality, we may assume that A is of the form 
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for some Ai > 0. Let A denote the set of constraints that are active at D* 



A {./:/,jD/, ; (114) 

Let i denote the constraint in A that weights y\ most heavily 

fiji 

i = arg max — — . 

j£A (JLj2 

We will show that D* is sum-rate optimal for the /x^-sum problem with dis- 
tortion di. Now D* is associated with a point (i?i,0) on the boundary of the 
/^-sum rate region. If this point is not sum-rate optimal for this problem, then 
it is possible to move a small distance to the left along the boundary of TZ* (di) 
and strictly decrease the sum rate. Since the distortion associated with fij 
varies continuously as one moves along this boundary, for a sufficiently small 
movement, none of the constraints in A c will be violated. On the other hand, by 
Lemma 9, none of the constraints in A will be violated either. This contradicts 
the assumption that D* is sum-rate optimal for the iVf -sums problem within the 
class T>q. It follows that D* is sum-rate optimal for the /J r sum problem. □ 

The previous two lemmas allow us to focus on the case in which there exists 
an optimizing D* in T>° G that meets at least two of the distortion constraints with 
equality. Our proof in this case parallels the proof of Theorem 1. In particular, 
we introduce a nonnegative vector /it* such that D* is sum-rate optimal for the 
/Lt*-sum problem. 

Lemma 13. Suppose D* G T>° G is sum-rate optimal within the class T>q 

f'r) flj ,/, vje {i,...,J} 

\ log W\ = inf { I log Tof : D e Vg and - dj Vi e {1, • • • ' J} } • 

(115) 

Let A denote the set of constraints that are active at D* 

A = {j : (i ' D ii = d , \. 

If |^4| ^ 2, then there exists i,j in A, i ^ j, and a nonnegative vector fi* such 
that 

1. D* is sum-rate optimal for the fi*-sum problem, 

\ log j§f| = inf log : D 6 V G and »* T D^ < /x* T £)*/x*| 

(116) 

2. M^ 1 ^* is nonnegative, where 

M 2 =[^ (117) 
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Proof. Any D € T>q can be written 



D 1 = Ky 1 + A 

for some diagonal and positive semidefmite matrix A. Thus the optimization 
problem in (115) can be written 

1 |.Kr x + A| 

min -log — i (118) 

2 \ K y \ ^ ' 

subject to nj [K^J 1 + A)' 1 < dj j G {1, . . . , J}, (119) 

where A ranges over all diagonal and positive semidefmite matrices. By hy- 
pothesis, this optimization problem is solved by 

A* = D* 1 -Ky 1 . 

Unfortunately, A* may not be regular [25, p. 309] for this optimization problem. 
This is an issue because the KKT conditions may not hold at a local minimum 
that is not regular [25, Example 3.1.1]. We proceed by using a generalization 
of the KKT conditions called the Fritz John conditions [25, Sec. 3.3.5]. The 
difference between the Fritz John conditions and the KKT conditions is that 
the Fritz John conditions include a scalar for the objective in addition to the 
multipliers for the constraints. The Lagrangian for this optimization problem is 

\ log lK L-i^ E "ite - ^^v 1 + A ) _1 ^) - E "•/•.,«><'.,• ( 12 °) 

I y I j=i j=i 

where the second summation handles the constraint that A must be positive 
semidefmite. Denote the coordinates of D* by 



D* 



dl 0*^/dJd* 



(121) 



By differentiating the Lagrangian with respect to the two diagonal entries of 
A and using the calculations in Appendix H, we can express the Fritz John 
conditions [25, Prop. 3.3.5] as 



J 

"0*1 = E v;(»jD* ei ) 2 + v* J+1 (122) 

i=i 
./ 

"o<3 - E ^{^D*e 2 f + v* J+2 (123) 

3=1 

for some fg, . . . , ^} +2 that are nonnegative but not all zero. Now since D* G T>° G 
by hypothesis, the constraint that A be positive semidefmite is not active at 
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A*. By complimentary slackness, this implies that v* J+l = v* J+2 = 0. Also by 
complimentary slackness, v* = if j ^ A, so it follows that 



Now if i/q were equal to zero, then we would have 

E^[(MjU*e 1 ) a + ( M ^e a ) 9 ]=0. 



(124) 
(125) 

(126) 



Since /itj is unit-norm for all j by assumption, (fij D* ei) 2 + (fij D* e 2 ) 2 must 
be positive for all j. Thus (126) would imply that v* = for all j in A. 
But this would contradict the condition that at least one of the dual variables 
i/q, . . . , v*j + 2 is nonzero. It follows that is positive, so we can divide through 
by Vq in (124) and (125) to obtain 



S = E ^CMj^eO 2 



(127) 
(128) 



Thus (d*, d 2 ) lies in the convex cone formed by the points 

{{^D*e 1 f, (/xj£>*e 2 ) 2 ) j e A. 

By Caratheodory's theorem for convex cones [24, Corollary 17.1.2], there exists 
i,j in A, i ^ j, such that (d*,d%) lies in the convex cone of those two points 
alone 



d\ = v l (nfD*e 1 ) 2 + P J -(/ijD*e 1 ) 2 
d* 2 = D l ( t ifD*e 2 ) 2 + D 3 (fijD*e 2 ) 2 . 



(129) 



Let Af 2 denote the 2x2 matrix 

M 2 



By swapping the roles of and /z^ if necessary, we may assume that |Af 2 | > 0. 
Let 



M = 



(130) 



By Lemma 4, ID* is sum-rate optimal for the /x*-sum problem, so it only remains 
to show that M 2 X fi* is nonnegative. 
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Consider the matrix 

b 



(tfD*e 2 ) 2 (pjD*e 2 ) 
In terms of the components of D* and M 2 , the determinant of D is 

\b\ = {^id\ + ^ 2 e*^d^ 2 ) 2 { Nl e*^d^ 2 + H2 d* 2 ) 2 



(131) 



- (n^dl +^ 2 0*^d*d*) 2 (^ 1 O*y/d*d* +[l l2 d* 2 ) 2 . (132) 

By expanding the products and using the fact that \M 2 \ > 0, one can show 
that \D\ > 0. In particular, the conditions in (129) can be written as 



b 1 



1 

W\ 



dt 

d* 2 

{^D*e 2 f 
-(Mf£>*e 2 ) 



-(Mj^ex) 2 
(tfD* ei ) 2 









d* 



Since vi and Vj are nonnegative, this implies that 

d* 1 ( l ifD*e 2 ) 2 > d* 2 (fijD* ei ) 2 
d* 2 ( fJ ,fD*e 1 ) 2 >d* 1 ( f ifD*e 2 ) 2 . 

Now (135) can be written 



(133) 
(134) 



(135) 
(136) 



\/dt(fiji0* \fd\d* 2 + n j2 d* 2 ) > y/d^ifijxdl + fi j2 
This can be rearranged to show that 

Vj2\/d 2 T > Hj\^/d\. 

Likewise, (136) implies 



Now M 2 1 fi* can be written 



M3V* 



1 



1 

MilMj2 — Mi2Mj'l 



Mj"2 -/ijl V « 2 

— M«2 Mil L V^l 

Mj2 - V,j\\fdl 



Mil V rf* — Mi2 V "2 



(137) 
(138) 

(139) 
(140) 



which is component-wise nonnegative due to (137), (138), and the nonnegativity 
ofdet(M 2 ). □ 

Lemma 14. Suppose Rz, d\, . . . , dj) is strict-sense achievable and there 
exists a D* in T>° G that is sum-rate optimal within the class T>q and meets at 
least two of the distortion constraints with equality. Then 



i?i+i? 2 > ^ogyS 



^sum( rf l 
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Proof. We give an abbreviated proof due to the similarity to the proofs of Lem- 
mas 5 and 6. From Lemma 13, there exists two constraint vectors (Jb i and fj,j, 
i 7^ j, and a nonncgative vector fi* such that 

(i) fifD*^ = d, 

(it) fijD* fij = dj 

(Hi) D* is sum-rate optimal for the /x*-sum problem, and 
(iv) M^ 1 ^* is nonnegative, where 

M 2 = [n l fij}. 



For in (-1,1), let 



Also define 



and 

R S um(&) 

= inf 



2 S |£>| 



ly/didj 



tyjdidj 



#coop(0) = 2 lo S 



(141) 
(142) 



DeV G and fi* T Dn* < fi* T M^ T DqM^ 1 fi* 



(143) 



Now fix some code that achieves (Ri, R 2 ,di, ■ ■ ■ ,dj). Let z n (i) denote the 
decoder's estimate of M 2 y n (i), which we may assume is the conditional expec- 
tation of Mly n (i) given the received messages. Let 



n 

n * — ' L 



(144) 



denote the average covariance matrix of M 2 y n — z n ■ Then 

D = M^DM^ 1 

is the error covariance matrix for the estimate of the source, y™. As in the proof 
of Lemma 5, D must be positive definite, and the sum rate of the code must 
satisfy (c.f. (33)) 



M 2 DM 2 \ 



\K y \\M 2 ? 
\D\ 



(145) 
(146) 
(147) 
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In particular, D must also be positive definite. Let us write it as 



D 



di 9^J didj 
9 J didj dj 



where di < di, dj < dj, and 9 £ (—1,1). Now define 



9 y didj 
\Jdidj ' 

and note that <fi is in (—1, 1). Then D ^ D^, so it follows that 

Rl + R 2 > l -\o g + |j ^ M2|2 =W0). 
1 \ u 4>\ 

On the other hand, 

n 

-£>[Qu*V« ~ H* T Mz T z n (i)) 2 ) = n* T M 2 T bM^»* 



< yu* T M 2 T D^M 2 X ^* 



(148) 
(149) 



i.e., this code achieves distortion fi* T M 2 T D r j > M 2 1 fi* for the /i,*-sum problem. 
Lemma 3 then implies that 



It follows that 



Now let 



Ri+R 2 >R S u m (<j>). 



i?i + i? 2 > inf max(i? coop (60,i? sum (60). 
i,i) 



if = mId*m 2 



(150) 



(151) 



which must have diagonal entries di and dj , respectively. Let us write D as 



D 



9*^d~d 3 



9* \Jdidj 



Note that we must have 9* £ (0, 1) since D is positive definite and both M 2 
and D* have positive entries. For 9 > 9* , we have 

max(R coop (6), R sum (9)) > R coop (9) > R coop (9*) = i log + ^j^ 1 ' (i 52 ) 



D 



2 g ID* I 



(153) 
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Since both components of M 2 1 /x* are nonnegative, it follows that 

is nondecreasing in 9, which implies that R sum (-) is nonincr easing. Thus if 

9 < §*, 

max( J R COO p(0),4um(0)) > Rsum(9) > Rsnm(0*). 

But observe that at 9* , i? SU m(') satisfies 

R sum (9*) = inf jllogJ^ji : D e 2? G and M * T D/x* < ^ T M^b*M 2 V | 

(154) 



and (151) implies that 



M^lfM^ 1 = D* 



Since D* solves the /x*-sum problem, this implies 

Wr) = iio g jg!i. 

It follows that 

e m£ i)m ax(i? coop (0),i? sum (0)) = ilogl^||. (155) 

Combining this with (150) yields the desired conclusion. □ 

We are now in a position to complete the proof of Theorem 2. 

Proof of Theorem 2. We only need to show that the rate region is contained in 
lZ* um (di, . . . , dj). Lemmas 11, 12, and 14 together imply that if 

(i?i,i?2, di, . . . , dj) 

is strict-sense achievable, then 

Rx +R 2 > inf jl log ^ :DeV G and fij Dfi z < d, Vj e {1, . . . , J}| . 

(156) 

It is readily verified that the right-hand side is lower-semicontinuous in d\, ■ ■ ■ , dj. 
It follows that (156) also holds if i?2, d\, . . . , dj) are achievable. This implies 
the desired conclusion. □ 
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G Converse for Many Sources 



In this appendix we prove Theorem 3. Recall we are assuming that the covari- 



ance matrix K y has the form 



Ky = 



1 P 
P 1 

P P 



P 
P 

1 



for some < p < 1 and all of the distortion constraints are equal to d. Our goal 
is to show that the minimum sum rate, i?* um (<i), equals 



i i log J^ji :DeV G and ejDe t < d V £ € {1, . . . , L) 
and that this infimum is achieved by a D € T)q of the form 



D 



k: 



XI 



(157) 



(158) 



for some A > 0. The conclusion is obvious if d > 1. so we will assume that 
< d < 1. Using the matrix inversion lemma [26, p. 50], the matrix inversions 
in (158) can be computed explicitly 



D 



1 



l + A(l-p) 



pll 1 



l-p + \(l-p)(l-p + P L) 



(159) 



It follows that there is a unique D £ T>q of the form in (158) with all of the 
diagonal entries equal to d. Let us call this matrix D*. Note that D* must be 
of the form 

"1 0* ... 0* 
9* 1 ... 6* 



D 



1 



(160) 



for some 9* > 0. Since D* must be positive definite, it follows that 6* < 1. 
Now the inequalities 

«L,M < tof ji '»Sn§T : D e V B and ejDe, < d W € {1, . . . ,£} J (161) 

aioglgff (162) 

are clear, so it suffices to show that 
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For 9 in (-1/(L - 1), 1) let 

D„ = d 



1 



9 9 ■■■ 1 

and note that Dg is positive definite for each 9. Then define 



-s^ffi- V'g -«+«) ■ 



Next consider the problem of reproducing the sum of the sources, l T y, at 
the decoder. By following the proof of Lemma 8, one can show that this problem 
is equivalent to the CEO problem 

y c = x + n e i G {1, ...,£} 

where x, rii, . . . are zero-mean, Gaussian, and independent, and x has vari- 
ance p. It follows from existing results that the separation-based scheme achieves 
the entire rate region, and in particular it is sum-rate optimal [8, 10]. Thus the 
sum rate for the 1-sum problem with distortion constraint 1 T Del is given by 

Rsum(6) ■= inf {^ lo S^/ 1 D G Vg and lTD1 ~ lTDel ] ■ ( 164 ) 

In fact, the CEO results imply that the infimum in (164) will be achieved by a 
D G T>a of the form in (158). In particular, we have 

iW**) = ^l°gj§2p (165) 
Lemma 15. If (Ri, . . . , Rl, d, . . . , d) is strict-sense achievable, then 

L 

J2 R z> . i nf > max(i? coop (0),i? sum (0)). (166) 
s e(— i/(L— i),i) 

Proof. By hypothesis, there exists a code (/} , . . . , , f± \ ■ ■ ■ , <Pl ) such 
that 

Rt>- log Mi n) for all £ in { 1 , . . . , L} 

n 

n 

d> - Y, E M(i)-ye(i)) 2 ] foralHin{l,...,L}. 

2 — 1 

From this code, we construct a new code with block length N := L\ ■ n by time- 
sharing among all L\ permutations of the sources. For this new code, the rates 
are symmetric 

1 L 1 

i=l 
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and the distortion still satisfies 

N 

Let 

N 



1 N 

rf>^E E[(yf(i) - y? (i)) 2 ] for all I in {1, ... , L}. 



i=l 

denote the error covariance matrix of the code. By the symmetry of the time 
sharing. D must have the form 



D = d 



1 



f 



for some d < d. Following the calculation at the beginning of the proof of 
Lemma 5, one can show that D is positive definite, which implies that —1/(L — 
1) < cj) < 1, and 



\_Ky_ 
\D\ 



But D -< Dfi, so this implies 

L 



(167) 



Since the code has error covariance matrix D, the distortion it achieves for the 
1-sum problem is at most 1 T Dl < l T D<f,l. It follows that 



Combining this with (167) gives 



^2,Ri > max(i? sum (0),i? coop (</>)). 



The conclusion follows by taking the infimum over <fi in (—1/(L — 1), 1) □ 

Next we evaluate the infimum in (166). Recall that 6* is defined by (160). 
Lemma 16. 

a , M , max(i? coop (6»),i? sum (6l)) = ilogf^}. 
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Proof. By differentiating, one can verify that i? C oop(0 is nondecreasing on (0, 1 ). 
Then if 9 > 9*, we have 



max(i 1 , coop (0),i? sum (0)) > R coop (9) > R coop (9*) = i log & 
On the other hand, if # < then since i? S um( - ) is nonincreasing 

max(i? coop (#),i? smn (0)) > i?sum(e) > i?sum(^*) = \ log j^fj 

where we have used (165). 



(168) 



(169) 



□ 



Theorem 3 now follows from a continuity argument similar to the one used 
in the proof of Theorem 1 . 



H Derivatives 

Let D(Ai, A 2 ) denote the matrix in Vq defined by 



D(Ai,A 2 )= [K- 1 



Ai 
A 2 



In this appendix we compute the derivatives of 

log| J D(A 1 ,A 2 )| 

and 



H T D(X 1 ,X 2 )n 



with respect to Ai and A2. Write 

A = 



Ai 
A 2 



Then we have 

01og|£»(Ai,A 2 ) 

axi 



(170) 



log|l<:- 1 +A + ( 5e 1 ef|-log|iir- 1 +A| 
hm - - 

5^0 5 

.. log|/ + <5e 1 efD(A 1 ,A 2 )| 
hm 

<5->o S 



= — lim 

5^0 



log(l + 5ef£>(Ai,A 2 )ei) 



Similarly, 



= -log(e)efD(Ai,A 2 )ei 
d\og\D(X 1 ,X 2 )\ 



dX 2 



log(e)e£ D(Xu A 2 )e 2 . 



(171) 

(172) 

(173) 
(174) 

(175) 
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Likewise, we have 

dfi T D{X 1 ,X 2 )(J- . 17g , 

= lim i >— (177) 

= - lim /x T (Z?(Ai, A a )ei(l + «Jef D(Ai, A 2 )e 1 )- 1 ef D(Ai, A 2 )) /i (178) 

= -( A t T Z)(A 1 ,A 2 )e 1 ) 2 (179) 

where to obtain the second equation we have used the matrix inversion lemma. 
Similarly, 

dfi T D(X u X 2 )fi 



9A 2 



( f i 1 D(X 1 ,X 2 )e 2 y. (180) 
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Figure 1: The two-encoder source-coding problem. 
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Figure 2: A natural architecture that separates the analog and digital aspects 
of the compression. 





5r 




4.5- 




4- 




3.5- 










a 


3- 














a 

a 


2.5- 


CO 




■— I 




s 

IN 


2 - 




1.5- 




1 - 




0.5- 




oL 








0.5 1 1.5 



7Z5(4) ' 

2 2.5 3 3.5 4 

i?i (Bits per sample) 



4.5 5 



Figure 3: The rate region for p = 0.9 and d\ = d% = 0.05. 



50 




0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 



Figure 4: Dependence of the product did 2 on the error correlation coefficient 9 
for p = 0.9. 
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Figure 5: R CO op(-) and Rsum(-) for the case p = 0.9 and d\ = d 2 = 0.05. The 
plot for -Rsum(') was generated using the convex optimization formulation of the 
sum rate for the (i-sum problem given in Appendix C. The circled point at 
which the two functions intersect is the min-max and equals the sum rate. 
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Figure 6: Graphical depiction of the lower bound argument. 
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Figure 7: An example illustrating the definitions of W, X, Y, and Z in Ap- 
pendix E. 
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Figure 8: Illustration of iVf -sums achievability argument. 
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Figure 9: Setup for the many sources problem. 
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